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Background The world is short of 7.2 million health— care workers 
and this figure is growing. The shortage of teachers is even greater, 
which limits traditional education modes. eLearning may help over- 
come this training need. Offline eLearning is useful in remote and 
resource— limited settings with poor internet access. To inform invest- 
ments in offline eLearning, we need to establish its effectiveness in 
terms of gaining knowledge and skills, students' satisfaction and at- 
titudes towards eLearning. 

Methods We conducted a systematic review of offline eLearning for 
students enrolled in undergraduate, health-related university de- 
grees. We included randomised controlled trials that compared of- 
fline eLearning to traditional learning or an alternative eLearning 
method. We searched the major bibliographic databases in August 
2013 to identify articles that focused primarify on students' knowl- 
edge, skills, satisfaction and attitudes toward eLearning, and health 
economic information and adverse effects as secondary outcomes. 
We also searched reference lists of relevant studies. Two reviewers 
independently extracted data from the included studies. We synthe- 
sized the findings using a thematic summary approach. 

Findings Forty-nine studies, including 4955 students enrolled in 
undergraduate medical, dentistry, nursing, psychology, or physical 
therapy studies, met the inclusion criteria. Eleven of the 33 studies 
testing knowledge gains found significantly higher gains in the eL- 
earning intervention groups compared to traditional learning, where- 
as 21 did not detect significant differences or found mixed results. 
One study did not test for differences. Eight studies detected signif- 
icantly higher skill gains in the eLearning intervention groups, whilst 
the other 5 testing skill gains did not detect differences between 
groups. No study found offline eLearning as inferior. Generally no 
differences in attitudes or preference of eLearning over traditionaf 
learning were observed. No clear trends were found in the compar- 
ison of different modes of eLearning. Most of the studies were small 
and subject to several biases. 

Conclusions Our results suggest that offline eLearning is equivalent 
and possibly superior to traditional learning regarding knowledge, 
skills, attitudes and satisfaction. Although a robust conclusion can- 
not be drawn due to variable quality of the evidence, these results 
justify further investment into offline eLearning to address the glob- 
al health care workforce shortage. 
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The world is short of 7.2 million health— care workers and 
this figure is growing [ 1 ] . The shortage of teachers is even 
greater, which limits traditional education modes. Health 
workers are fundamental to ensuring equitable access to 
health services and achieving universal health coverage. In 
2006, the World Health Organization (WHO) reported that 
fifty— seven countries were facing critical health workforce 
shortages due to lack of adequate training or migration 
(brain drain) [2] . Although major progress has been made 
to tackle the earlier estimated shortage of 4.3 million health 
workers globally [2] , the numbers of health workers still 
need to be scaled up considerably [3] to achieve the Mil- 
lennium Development Goals [4] . 

eLearning might help to address the training need for 
health workers. Many universities are already using eLearn- 
ing to support traditional campus-based education or en- 
able access to distance or flexible learning. Perceived ad- 
vantages include reduction of the costs associated with 
delivery of educational outcomes [5] , improving scalabil- 
ity of educational developments [6] , increasing access and 
availability to education by breaking down geographical 
and temporal barriers and allowing access to experts and 
novel curricula [7]. 

eLearning is "an approach to teaching and learning, repre- 
senting all or part of the educational model applied, that is 
based on the use of electronic media and devices as tools 
for improving access to training, communication and in- 
teraction and that facilitates the adoption of new ways of 
understanding and developing learning" [8] . It does not 
only differ from traditional learning (ie, face— to— face learn- 
ing that takes place in a classroom environment) in the me- 
dium by which learning is delivered [9] , but also affects the 
teaching and learning approaches used. eLearning can take 
the form of a full eLearning approach, which is entirely 
driven by technology, or be a mix of the traditional and 
fully computer-based methodologies (blended learning). 
Blended learning might be more suitable for health care 
training because of the need to combine hands-on skills- 
based training at practical level as well as self-directed 
learning [10-14]. 

The United Nations (UN) and the WHO regard eLearning 
as a useful tool in addressing education needs in health 
care, especially in developing countries [15,16] where the 
worst health workforce shortages occur [2]. Currently, the 
most renowned eLearning initiatives focus on the online 
delivery of and online interaction with the learning mate- 
rials. However, in resource-limited settings this approach 
is often not possible. Only 31% of the population had in- 
ternet access in developing countries in 2013 [17] . Because 
network connectivity and bandwidth availability are key 
obstacles to effective delivery of eLearning content 
[9,18,19], a partially or completely offline eLearning ap- 



proach may be more suitable in rural and/or developing 
areas. Offline computer— based eLearning delivered through 
eg, a CD-ROM or USB stick, for example, can be particu- 
larly efficient in increasing the accessibility quality and 
availability of health related education within limited costs 
in remote areas with limited teaching staff, equipment, 
technological infrastructures and resources available. As- 
sessing the effectiveness of these interventions for health 
professional education could provide an evidence base to 
guide and inform future projects and policies aimed at ad- 
dressing the global shortage of health workers. 

To our knowledge only 2 systematic reviews of randomised 
controlled trials (RCTs) assessing the effectiveness of offline 
eLearning have been conducted so far [20,21]. Both re- 
views were published over a decade ago. Besides, they only 
focused on dentistry [21] and medical [20] education. 

We conducted a systematic review to compare the effective- 
ness of offline eLearning with traditional learning in terms 
of gaining knowledge and skills, students' satisfaction and 
attitudes towards eLearning. 

METHODS 

We conducted a systematic review following the Cochrane 
methodology [22]. 

Search methods for identification 
of studies 

Electronic searches. We limited our electronic searches to 
records published on or after the year 2000 in order to 
highlight recent developments. 

We developed a search strategy for MEDLINE (OvidSP) us- 
ing a combination of keywords and MeSH terms that cap- 
tured the types of intervention and the types of participants 
under evaluation in this systematic review (Table 1). We 
adapted the search strategy for use in EMBASE (OvidSP), 
PsycINFO (Ovid SP), Cochrane Central Register of Con- 
trolled Trials (CENTRAL), Web of Science, and Education- 
al Resources Information Center (ERIC) (ProQuest). 

Where available, we used validated methodological filters 
to limit our searches to Randomised Controlled Trials 
(RCTs) and cluster RCTs (cRCTs). We ran the searches in 
August 2013. 

Searching other resources. We checked reference lists of 
the included studies and systematic reviews of the literature 
identified by our electronic searches for additional studies. 

Inclusion criteria 

Types of studies and participants. We included studies 
published in any language on students of (i) undergradu- 
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Table 1 . Search strategy for use in MEDLINE (Ovid SP)* 
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and Ovid MEDLINE® 1946 to Present. Date of search: 16 August2013 
09:53. Limits: Year — 2000. Eilter: Cochrane Highly Sensitive Search Strat- 
egy for identifying randomized trials in MEDLINE: sensitivity— maximiz- 
ing version (2008 revision); Ovid format. 



ate, health-related university degrees; or (ii) basic, health- 
related vocational training programmes. We defined un- 
dergraduate education or basic vocational training as any 
type of study leading to a qualification that: (i) is recognised 
by the relevant governmental or professional bodies of the 
country where the studies were conducted; and (ii) entitles 
the qualification-holder to apply for entry level positions 
in the health care workforce. For this reason, graduate 
medical education courses from the USA were included. 

We considered studies on candidates for and holders of the 
qualifications listed in the Health Field of Education and 



Training of the International Standard Classification of Ed- 
ucation (ISCED-F) [23], except studies on students of tra- 
ditional and complementary medicine. We hence included 
students reading dental studies, medicine, nursing and 
midwifery, medical diagnostic and treatment technology, 
therapy and rehabilitation, or pharmacy. Medicine and 
dentistry were classified under the umbrella term allied- 
health professions. 

Types of intervention. First, we conducted a systematic 
mapping of the types of technologies used by the included 
studies to deliver the learning materials, through which we 
identified 6 broad categories of eLearning interventions, 
based on the technologies employed: (1) Offline comput- 
er—based eLearning, (2) Online and local area network- 
based eLearning, (3) Psychomotor skills trainer, (4) Virtu- 
al reality environments, (5) Digital game-based learning 
and (6) mLearning. 

We allocated each included study to the category that fit- 
ted the study best (definition of these categories is available 
in Online Supplementary Document). 

We only included studies in which offline eLearning inter- 
ventions were used to deliver the learning content, which 
we defined as standalone applications where internet or in- 
tranet connections were not required for the delivery of the 
learning activities. The eLearning software and interactions 
thus run entirely on a PC or laptop. Delivery channels of 
the software could be via CD-ROM or a USB memory stick. 
If the delivery mode of the software was based on a net- 
worked connection but the learning activities did not rely 
on this connection — ie, a replacement delivery channel 
could easily be identified with low efforts/costs, without 
any restrictions on original intended usage — then this is 
also an offline intervention. 

Only studies that compared eLearning or blended learning 
methods to: (i) traditional learning; (ii) an alternative eL- 
earning or blended learning method; or (iii) no interven- 
tion were eligible for inclusion. These studies could either 
be studies where eLearning was the sole means by which 
the intervention was delivered or where eLearning was part 
of a complex, multi— component intervention. 

Types of outcome measures. To be eligible for inclusion, 
studies had to report at least 1 of the following primary or 
secondary outcomes: 

Primary outcomes. Primary outcomes were: (1) Students' 
knowledge, measured using any validated or non-validat- 
ed instrument (eg, pre- and post-test scores, grades, per- 
ceived knowledge survey scores); (2) Students' skills, mea- 
sured using any validated or non— validated instrument (eg, 
pre- and post-test scores, time to perform a procedure, 
number of errors made whilst performing a procedure, per- 
ceived up-skilling); (3) Students' satisfaction and attitudes 
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towards eLearning, measured using any validated or non- 
validated instrument (eg, self-efficacy, satisfaction, accept- 
ability). 

Secondary outcomes. Secondary outcomes were: (f) 
Health economic properties of the interventions (eg, imple- 
mentation cost, return on investment); (2) Adverse and/or 
unintended effects of eLearning (eg, potential feelings of 
depression and loneliness, dropout risks [24] and "com- 
puter anxiety" [25]). 

We only considered studies to have measured students' sat- 
isfaction and attitudes towards eLearning if they met all of 
the following criteria: (i) they compared the differences be- 
tween intervention and control groups for these outcomes; 
(ii) the content of the survey questionnaires related to the 
teaching method (ie, eLearning method, blended learning, 
or traditional learning); and (iii) the adjectives used in the 
survey questionnaires accurately described attitudes and/ 
or satisfaction. 

Study selection and data collection 

The study selection process is summarised in the PRISMA 
flow diagram (Figure 1). In brief, we screened the titles 
and abstracts of the citations identified by our electronic 



and manual searches to identify potentially relevant stud- 
ies, of which we assessed the full-text report to ensure they 
meet the inclusion criteria we specified. Review authors 
completed these tasks independently and met to compare 
their results and reach consensus. 

Every selected study was allocated to a pair of review au- 
thors, with ten review authors participating in total. Each 
review author independently extracted data from the in- 
cluded studies using the structured data extraction sheet 
shown in Online Supplementary Document. 

Each pair of reviewers compared their completed data ex- 
traction forms and any discrepancies between review au- 
thors' results were resolved through discussion; if no agree- 
ment could be reached, a third review author acted as an 
arbiter. Because ten review authors participated in the data 
extraction process, some categories were interpreted differ- 
ently by some reviewers. Therefore, 3 reviewers went over 
the entire data extraction again to ensure uniformity. 

We contacted authors of studies containing incomplete 
data to request the missing information. Some authors did 
not reply to our request for additional information, whilst 
other authors did not know the answer to our questions. 
For a single study, the response obtained from the author 
resulted in the subsequent exclusion of 
the study from the systematic review. 



12208 records identified from database 
searching 



9091 records after removal of duplicates 



309 records included for full text reading 



207 eligible articles were included 



59 eligible articles were included 



47 articles (49 trials] had relevant 
interventions and outcomes and were 
included in the analysis 



> 3117 duplicates excluded 



8780 records excluded after 
screening of title and abstract 



102 records excluded after full text 
reading 



70 classified as online eLearning 
78 classified as simulation based 
eLearning 



12 articles excluded from offline 
eLearning. 

Reasons for exclusion: 

" One moved to mLearning 

" One had participants that were 

not undergraduate students 
" One was a duplicate publication 
" Five studies had a design that was 

not RCT or cluster RCT 
" Four were published before 2000 



Figure 1 . Flowchart of the studies included in the 



Assessment of risk of bias in 
included studies 

During the data extraction process, we 
assessed the risk of bias at the outcome 
level using tools recommended by the 
Cochrane Collaboration [22]. ForRCTs, 
we did so across the domains of (1) ran- 
dom sequence generation, (2) allocation 
concealment, (3) blinding of partici- 
pants and personnel, (4) blinding of 
outcome assessment, (5) incomplete 
outcome data, (6) selective outcome re- 
porting, and (7) other bias including the 
comparability of intervention and con- 
trol group; characteristics at baseline; 
validation of outcome assessment tools; 
reliability of outcome measures; and 
protection against contamination. 

We assessed the risk of bias for cRCTs 
across the domains of (1) recruitment 
bias, (2) baseline imbalances, (3) loss 
of clusters and (4) incorrect analysis. 

For each study, 2 reviewers indepen- 
dently categorised each domain as low, 
high or unclear risk of bias. 
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Summarising the data 

We qualitatively compared the characteristics of the par- 
ticipants and of the interventions between the included 
studies to determine the feasibility of conducting a meta- 
analysis. Because of substantial clinical, educational, con- 
tent and methodological heterogeneity we did not conduct 
a meta-analysis. Instead, we adopted a thematic summary 
approach [26]. 

RESULTS 

The study selection process is depicted in Figure 1 . The 
initial search yielded 12 208 records. After removing 3117 



duplicate records using EndNote X5, we screened the titles 
and abstracts of 909 1 records (see Online Supplementary 
Document for a detailed description). After this initial 
screening, we excluded 8780 records. We retrieved the full 
text reports for the remaining 309 records and assessed 
them for eligibility. Of these, we excluded 102 articles that 
did not meet the eligibility criteria (Figure 1). 

Forty— seven [27—73] of the remaining articles complied 
with the term offline eLearning. 

Two [54,70] articles reported results of 2 separate cRCTs 
that were analysed separately, and 2 articles [43,74] report- 
ed results from the same study. This resulted in a total num- 
ber of evaluated studies of 49 (Table 2). 



Table 2. Summary of findings for the 40 studies comparing offline eLearning with traditional learning 



Study 


Discipline 


Knowledge 


Skills 


Attitude 


Satisfaction 


No. of Intervention delivery approach 
participants 


Characteristics 


Ackermann 2010 [27] 


Medicine 


E 








19 


Full eLearning 


CG: Lecture 
IG: CD-ROM 


Amesse 2008 [28] 


Medicine 


E 








36 


Full eLearning 


CG: Paper based tutorial 
IG: Computer based tutorial 


Armstrong 2009 [29] 


Medicine 


NS 








21 


Full eLearning 


CG: Lecture 

IG: Interactive slideshow 


Bains 2011 [30] 


Dentistry 


NS 




E 




90 


IG 1: Full eLearning 
IG 2: Blended learning 
IG 3'. Blended learning 


CG: Teacher— led tutorial 
IG 1 : Online tutorial only 
IG2: Online tutorial only, then 
teacher— led tutorial 
IG3: Teacher — led tutorial, then 
online tutorial only 


Bloomfield 2010 [31] 


Nursing 


NS 


M 






223 


Full eLearning 


CG: Lecture/video/practice 

IG: Computer module including 

video 


Boet 2010 [32] 


Medicine 




M 






42 


Blended learning 


CG: Lecture 

IG: Lecture + CD-ROM 


Bogacki 2004 [33] 


Dentistry 


NS 








45 


Full eLearning 


CG: Lecture 

IG: Computer program 


Bradley 2005 [341 


Medicine 


NS 




NS 




168 


Full eLearning 


CG: Workshops 

IG: Workbook + CD-ROM 


Davis 2008 [351 


Nursing 


NS 








179 


Blended learning 


CG: Lecture 

IG: Digital recording + PowerPoint 
slides + Internet links 


Feeg 2005 [361 


Nursing 


E 








91 


Blended learning 


CG: Journal article 

IG: Journal article + CD 


Gelb 2001 [37] 


Medicine 


NS 








107 


Full eLearning 


CG: Printed tutorial 
IG: Computer tutorial 


Glicksman 2009 [38] 


Medicine 




E 


E 




47 


Full eLearning 


CG: Article 

IG: Computer module with article 


Goldsworthy 2006 [39] 


Nursing 


E 








25 


Full eLearning 


CG: Paper— resources 
IG: PDA-based resources 


Green 2011 [40] 


Medicine 


E 






E 


121 


Full eLearning 


CG: Paper— based resources 
IG: Computer program 


Holt 2001 [41] 


Medicine 


NS 








108 


Full eLearning 


CG: Lectures 

IG: Computer— based lectures 


Howerton 2002 [42] 


Dentistry 


NS 








59 


Blended learning 


CG: Lectures 
IG: CD-ROM 


Jeffries 2003 [45] 


Nursing 


NS 


NS 


NS 


NS 


73 


Full eLearning 


CG: Self— study module + instructor 

led demonstration 

IG: Self— study module + CD 


Kim 2003 [48] 


Nursing 


NS 


NS 


NS 


E 


75 


Blended learning 


CG: Printed material 

IG: Computer-based material 


Kong 2009 [49] 


Medicine 


E 


E 




E 


90 


IG 1: Blended learning 
IG 2: Other learning 


CG: Didactic teaching 

IG 1 : Paper— based Problem Based 



Learning 

IG 2 : Computer— based Problem 
Based Learning 
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Table 2. Continued 


Study 


Discipline 


Knowledge 


Skills Attitude Satisfaction 


No. of Intervention delivery approach 

PARTICIPANTS 


Characteristics 


Kurihara 2004 [50] 


Medicine 


E 


E 


59 IG 1: Full eLearning 

IG 2: Blended learning 
IG 3: Traditional learning 


CG: Textbook only 

IG 1: Computer program only 

IG 2: Textbook + Computer program 

IG 3: No intervention 


Lira 2013 [51] 


Medicine 


M 




68 Blended learning 


CG: Lecture 

IG: Lecture + PDF article 



CG: Paper cases, textbook + optional 
lecture 

IG 1: Computer— based cases, 
textbook + optional lecture 
IG 2: No cases, optional text- 
book + lecture 



McDonough 2002 [53] 


Medicine 


NS 






Blended learning 


CG: Lecture + tutorial 

IG: Lecture + computer tutorial 


McMullan2011a t [54] 


Nursing 


E 


NS 


E 48 


Full eLearning 


CG: Paper hand-out 
IG: Computer program 


McMullan 2011b [54] 


Nursing 


E 


E 


50 


Full eLearning 


CG: Paper hand-out 
IG: Computer program 


Miedzybrodzka 2001 [55] Medicine 


NS 


NS 


48 


Full eLearning 


CG: Lecture 














IG: Computet piogram 


Nance 2009 [57] 


Dentistry 




NS E 


73 


Full eLearning 


CG: Paper hand-out + laboratory 

session 

IG: DVD only 


IN Ola ZUU5 IjoJ 


in — : — 

Medicine 


E 




85 


Full eLearning 


CG: Lectures + practical sessions 
IG: Lectures (optional} + computer- 
ised sessions 


Perfeito 2008 [60] 


Medicine 


NS 






Full eLearning 


CG: Lectuie 

IG: Computer program 


LJayumi l\Jv^ [0j\ 


Medicine 






99 


IG 1: Traditional learning 
IG 2' Full eLearning 
IG 3. Blended learning 


CG: No intervention 

Tfr 1 ■ Tpvt mnrliilf 5 

1 V_J ± . 1CAL 111L.IV_.L LI lt_ 

IG 2: Computer program 

T C -v ^ " ~Vf> V 1 TT"1 C\f\ 1 1 J 5 -X- fOTVI T^l 1 t^T 
1VJT 1CAL IIIIJCIUIC T CU111LJ ULE1 

I ' ! i am 


Roppolo 2011 [64] 


Medicine 




E 


180 


IG 1: Blended learning 

Tf-r J ' T^lf^nH^H m (T 
Ivjr z_ . UltllCltvl ICiAllllllti 


CG: Instructor and video based 

LUU1X \,COtilllLlVCy ^ 1115L1 ULLU1 ICCl 

course (practical) 
IG 1 : Online course (cognitive) + 
DVD — based course (practical) 
IG 2: Online course (cognitive) + 
Facilitator based practice (practical) 


Seabra 2004 [65] 


Medicine 


NS 




60 


Full eLearning 


CG: Lecture 

IG: Computer program 


Shomaker 2002 t [66] 


Medicine 


NS 




DNT 94 


IG 1: Full eLearning 
IG 2: Blended learning 


CG: Lectures, texts+slides 

IG 1: Computer program + texts 

IG 2: All of the above 


Solomon 2004 [67] 


Medicine 


NS 






Full eLearning 


CG: Lecture 
IG: CD-ROM 


Vichitvejpaisal2001 [69] 


Medicine 


M 




80 


Full eLearning 


CG: Textbook 

IG: Computer program 


Vivekananda— Schmidt 


Medicine 




E NS 


105 


Full eLearning 


CG: No CD-ROM 


2005a [70] 












IG: CD-ROM 


Vivekananda— Schmidt 


Medicine 




E E 


156 


Full eLearning 


CG: No CD-ROM 


2005b [70] 












IG: CD-ROM 


Weih2008 [71] 


Medicine 
and 


NS 




E 101 


Full eLearning 


CG: Lecture 

IG: Lecture + CD-ROM 




Psychology 










Williams 2001 [72] 


Medicine 


NS 




163 


Full eLearning 


CG: Lecture 

IG: Computer program 


Xeroulis 2007 [73] 


Medicine 




E 


60 


IG 1: Blended learning 


CG: No intervention 



IG 2: Traditional learning IG 1: Computer— based video 
IG 3: Traditional learning IG 2: Concurrent feedback during 
practice 

IG 3: Summary feedback after 

practice 

E — Results favoured computer— based eLearning over traditional learning, NS — No significant difference between eLearning and traditional learning, M 
— Mixed results, T — Results favoured traditional learning over computer— based eLearning, DNT — Difference not tested, CG — Control group, IG — In- 
tervention group 

^Knowledge improvement in the two eLearning groups as well as the traditional learning group, whereas the control group that received no interven- 
tion, showed minimal improvement. 

fin the cRCT by McMullan 2011 [55], the results for satisfaction were pooled for the two cohorts (McMullan 2011a and McMullan 201 lb) and the re- 
sult presented for McMullan 2011a therefore also includes students from the McMullan 2011b cohort. 
T^For students' satisfaction, no clear trends in terms of one intervention group being superior to another. 
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Included studies 

The 49 included studies were either parallel RCTs or cRCTs 
published in peer-reviewed journals between 2001 and 
2013. There were no clear trends in terms of increase in 
publication of offline studies in the time period investigat- 
ed. Thirty-five studies [27-29,32,34,37,38,40,41,44, 
46,47,49-53,55,5,58-70,72,73] investigated eLearning in 
the field of medicine, 8 in the field of nursing [31,35,36, 
39,45,48,54] and 4 in the field of dentistry [30,33,42,57] . 
One article [71] focused on both medicine and psychology 
whereas another [43] focused on medicine, dentistry and 
physical therapy at the same time. 

Participant characteristics 

The total number of participants included across all trials 
was 4955. The smallest study included 8 participants in 
the control and 8 participants in the intervention group 
[59] . The study with the largest control group had 177 par- 
ticipants [58] , while the largest intervention group had 113 
participants [31]. Most studies were conducted among un- 
dergraduate university students apart from 2 studies 
[31,36] that investigated the effect of offline eLearning for 
vocational training. Eleven studies that specified the age of 
the students. In the control groups, the mean age of par- 
ticipants ranged from 22.4 [30] to 30 years [35] . The mean 
age of participants in the control group was comparable, 
ranging from 21.8 [30] to 30 years [35]. 

Intervention characteristics 

Forty studies [27-42,45,48-55,57,58,60,63-67,69-73] 
compared eLearning to traditional learning and 9 studies 
[43,44,46,47,56,59,61,62,68] compared one mode to an- 
other mode of eLearning. The shortest duration of exposure 
was 20 minutes [47] and the longest was 1 year [58] . 

Most of the studies (42 out of 49; 86%) were 
conducted in high-income countries, and 13 of 
these [28,33,36,37,40,42,45,47,57,64,66-68] 
in the USA. The remaining 5 studies were con- 
ducted in low- and middle— income countries: 1 
[69] in Thailand; 1 [49] in China; and 1 
[51,60,65] in Brazil. Figure 2 shows the distri- 
bution of the countries in which the studies were 
conducted. 



The majority of the studies used programs that 
run on PCs or laptops [27-38,40-73]. One 
study [39] investigated the use of a Personal 
Digital Assistant (PDA), which is a small por- 
table electronic device that can be regarded as 
the predecessor of a computer tablet and smart- 
phone, with PDFs from Elsevier. Sixteen studies 
delivered the eLearning intervention to the stu- 



dents on a CD-ROM. [27,28,32-36,42,45,48,49,60, 
67,70,71]. The eLearning software and material used in 
the remaining studies were distributed via a variety of 
sources where specified: learning management systems 
such as WebCT Blackboard [30,43], DVDs [30,57,61], 
the internet [29,47,51], stored on a computer [53,58,69] 
or for 1 study [39] on a PDA, and email [56], Several in- 
terventions used standard vendor software such as Ado- 
be® [51,54], Macromedia AuthorWare® [66] and Micro- 
soft® PowerPoint® [29,36]. 

Primary outcomes 

Students' knowledge assessment. Overall, 40 [27- 
31,33-37,39-45,47-56,58,60-63,65-69,71,72] out of the 
49 studies looked at a knowledge based outcome. Nineteen 
of these [29-31,34,36,37,40,41,43,45,47,50,51,53,61,63, 
65,71,72] used only a multiple choice questionnaire 
(MCQ) to test students' knowledge and understanding, 
while another 9 studies [27,35,44,52,55,56,58,60,66] 
used a MCQ in conjunction with an additional testing 
method (eg, short answer questions or X-ray image inter- 
pretation). A further eleven studies [28,39,42,48,49,54,62, 
67—69] measured students' knowledge gain via other test- 
ing means including case analysis, X— ray image interpreta- 
tion and written exams. One study [33] did not specify 
which method was used to examine the participants' 
knowledge. 

Students' skills assessment. Skills were assessed in 16 
studies [31,32,38,45-50,57,59,63,64,70,73], the method 
of which was described in all but 1 study [49] . Ten of these 
[31,38,45,47,48,50,63,64,70] used a rating scale and/or 
checklists (eg, an Objective Structured Clinical Examina- 
tion - OSCE) to assess clinical skills. Three studies 
[46,59,73] used the Imperial College Surgical Assessment 
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Device and a checklist for the assessment. Another study 
[57] used a grading rubric to assess ability to carve teeth in 
wax. Another study assessing the ability to conduct orotra- 
cheal fibreoptic intubation [32] evaluated successful intu- 
bation in real time. 

Students' satisfaction and attitudes towards eLearning 
assessment. None of the studies assessed change in stu- 
dents' professional attitudes towards patients such as com- 
passion. 

Feedback from students assessed as their attitude towards 
the eLearning intervention was reported as an outcome in 
14 studies [30,34,38,45,48,52,54-57,61,70]. Participants 
were asked to provide ratings via Likert scales in 1 1 studies 
[34,48,52,54-57,61,70]. One study [38] used a question- 
naire and did not mention the use of Likert scales. In the re- 
maining 2 studies [30,45] , Likert scales were combined with 
another method, ie, focus groups in Bains et al. [30] and an 
additional questionnaire in Jeffries et al [45]. 

Students' satisfaction was considered as an outcome in 13 
studies [40,43,45,48,49,52-54,61,62,66,71]. Eight of 
these studies [40,43,49,53,54,61,62] specified that stu- 
dents' satisfaction was evaluated with Likert scale question- 
naires. The 5 remaining studies comparing students' satis- 
faction among the students [45,48,52,66,71] used different 
types of questionnaires without mentioning the use of Lik- 
ert scales. 

Secondary outcomes 

Health economic properties of the eLearning interven- 
tion. Health economic properties of the eLearning interven- 
tion were rarely mentioned in the included offline eLearn- 
ing studies. However, some of the studies addressed certain 
financial and resource related elements of eLearning. Davis 
et al. [35] mentioned that costs in producing the eLearning 
package were minimal and well within normal departmen- 
tal budgets for teaching undergraduates. Ackermann et al. 
[27] stated that effective learning can be performed with the 
use of few resources and provides a very economical mode 
for educating medical students. Bradley et al. [34] stated 
that the in-house development of the eLearning course ma- 
terial took 40 hours for the preparation of the course mate- 
rial, 10 hours to administer each semester and the internet 
site used for the eLearning group took 100 hours to devel- 
op. The eLearning course material also included a CD— ROM 
produced externally with an estimated cost of £ 30 per CD 
[34] . McDonough et al. [53] reported that it took local IT 
staff 4 hours to install the program on 20 PCs and that no 
maintenance was required after that point. Vivekananda— 
Schmidt et al. [70] stated that the costs of designing the eL- 
earning course were £ 11 740 (US$ 22045). Tunuguntla et 
al. [68] wrote in reference to comparing 2 different types of 
eLearning: "The cost ratio (measured in hours) for the mod- 



ule was about 2:3: about 72h for creation of the static graph- 
ics vs. 106h for the animations". 

Adverse or unintended effects of eLearning. Adverse or 
unintended events of the eLearning intervention were not 
reported in any of the studies. 

Excluded studies 

Initially 59 articles were categorised as offline eLearning 
studies. One study [75] was reclassified as mLearning be- 
cause lectures were viewed on an iPod [75] , and was there- 
fore excluded from this systematic review. Eleven studies 
[74,76-84] were excluded during the data extraction phase 
because they met 1 or more of the exclusion criteria. Four 
studies [77,78,83,85] of these were published before 2000. 
Five studies [79-82,84] were excluded because the study 
design was not a parallel or cRCT One study was excluded 
as the participants were not undergraduate students [76] . An 
additional study [74] was a secondary publication of a study 
that was already included [43] and information from the sec- 
ondary publication was merged with the included study 

Risk of bias in included studies 

The assessment of risk of bias is described in detail in On- 
line Supplementary Document. In summary, the major- 
ity of the included parallel RCTs were considered to be of 
low quality because of high risk of bias [28,31-34,36,38- 
44,47,50-52,56,57,62,63,66-69,71]. Only a few studies 
[27,37,46,48,49,53,55,58-61,65,66,72,73] were of high 
quality with none of the assessed categories rated as having 
a high risk of bias (Figure 3 and 4). In the majority of stud- 
ies at least 1 or more categories were classified as having an 
unclear risk of bias, especially with regards to the alloca- 
tion of participants to intervention groups. 

Effects of offline eLearning interventions 

The 49 randomized trials included in our review assessed 
the effectiveness of offline eLearning interventions in 
terms of knowledge, skills, attitudes and satisfaction. The 
findings were based on comparisons between offline eL- 
earning and traditional learning or between various 
modes of offline eLearning. A study may have compared 
more than 1 outcome between groups, and each outcome 
may have been assessed in multiple ways. For example, a 
study which compared students' acquisition of skills may 
have assessed skills in terms of the student's performance 
on a global rating scale, ability to perform a specific pro- 
cedure as well as the ability to comply with requirements 
in a checklist. As a result, the number of comparisons 
made across studies for a particular outcome may exceed 
the number of studies which reported that outcome. 

The studies were split into 2 research themes evaluating 
the impact of eLearning interventions for undergraduate 
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health care education: traditional learning vs offline eL- 
earning, and offline eLearning vs offline eLearning. 

Traditional learning vs offline eLearning 

Forty (82%) of the included studies [27-42,45,48-55,57, 
58,60,63—67,69-73] compared offline eLearning with tra- 
ditional learning. Please refer to Table 2 for a summary of 
findings of the individual studies, and to Online Supple- 
mentary Document for a further description of the nature 
of the interventions. 

Students' knowledge. Amongst the 40 studies which 
compared offline eLearning with traditional learning, 
knowledge was assessed in 33 (83%) studies [27-31,33- 
37,39-42,45,48-55,58,60,63,65-67,69,72], 5 of which 
were cRCTs [29,30,45,54]. Eleven (33%) studies 
[27,28,36,39,40,49,50,54,63] assessing knowledge gain 
demonstrated significantly higher knowledge gains for stu- 
dents assigned to offline eLearning compared to those ex- 
posed to traditional learning. Outcome measures for these 
studies were based on correct responses to questions which 
included true— false, multiple choice or fill in the blanks 
type of assessments. The sample size for these studies 
ranged from 19 to 225 with all but 4 studies [36,39,54] 
conducted on medical students. Seven of these studies used 
solely offline eLearning as the main intervention [27,28, 
39,40,54,63,84] whereas 4 used blended learning [36,42, 
49,58]. 

None of the included studies found greater gain in knowl- 
edge for the traditional learning group. 

Post— intervention knowledge was not significantly differ- 
ent between eLearning and traditional learning in 19(58%) 
of the included studies [29-31,33-35,37,41,42,45,48,53, 
55,60,65-67,71,72], 

Two (6%) studies [51,69] showed mixed results ie, favour- 
ing the intervention, control, or neither 1 depending on 
the specific indicator of knowledge being assessed. Anoth- 
er study [51] initially found no difference between the tra- 
ditional and offline eLearning groups, but statistically sig- 



Random sequence generation (selection bias) 
Allocation concealment (selection bias) 
Blinding of participants and personnel (performance bias) 
Blinding of outcome assessment (detection bias) 
Incomplete outcome data (attrition bias) 
Selective reporting (reporting bias) 
Other bias 



Figure 4. Risk of bias for 
each individual parallel 
randomised controlled trial 
(RCT) separately. 
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Figure 3. Overall risk of bias graph. 
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nificantly better post— test scores were seen in the offline 
eLearning group after 1 month. Another study [69] showed 
that students taught blood gas interpretation using a text- 
book had greater improvement from pre— test to post-test 
compared to those in the offline eLearning group, but after 
3 weeks the final test scores of both groups failed to show 
a significant difference between the 2 groups. 

In 1 (3%) study [52] knowledge was assessed, but not 
tested for statistically significant differences between the 
intervention groups. The study showed knowledge im- 
provement in the 2 offline eLearning groups as well as the 
traditional learning group, whereas the control group that 
received no intervention showed minimal improvement. 

Students' skills. Overall, 13 studies- 9 RCTs [31,32,38,48- 
50,57,63,73] and 4 cRCTs [45,64,70] measured skills as 
an outcome. 

Of the studies that evaluated differences in skills acquisi- 
tion, 8 (62%) [38,49,50,63,64,70,73] found significantly 
greater skills acquisition amongst students assigned to of- 
fline eLearning compared to those assigned to traditional 
learning. The range of skills assessed by these studies in- 
cluded performance in specific tasks, such as cardiopulmo- 
nary resuscitation, fiberoptic intubation and knot tying 
skills; performance in objective structured clinical exami- 
nation, as well self— efficacy assessments. The number of 
participants included in these studies ranged from 19 to 
354. All 8 studies [38,49,50,63,64,70,73] were conducted 
in medical students. Three (23%) studies [45,48,57] did 
not detect a significant difference in skill acquisition be- 
tween groups. 

None of the 13 studies demonstrated more favourable re- 
sults for traditional learning compared to offline eLearning. 

Results were mixed for 2 (15%) studies [31,32]. In 1 of 
these [31], testing hand washing skills of nursing students 
assigned to computer assisted vs conventional learning, 
skills were similar in both groups at the 2— week follow— up 
but were in favour of the intervention group at the eight- 
week follow-up. In the other study [32] that focused on 
intubation skills, successful intubation was more common 
in the offline eLearning group compared to the traditional 
group whereas there was no statistical significant difference 
in the checklist and global rating scale assessment of intu- 
bation skills. 

Students' satisfaction and attitudes towards eLearning. 

Twelve studies - 6 RCTs [34,38,48,52,55,57] and 6 cRCTs 
[30,45,54,70] - assessed attitudes towards the eLearning 
the intervention, primarily through Likert scale surveys. 

Five (42%) studies [30,38,54,57,70] found more favour- 
able results for students assigned to eLearning compared 
to traditional learning. 



Six (50%) studies [34,45,48,54,55,70] did not detect a sta- 
tistically significant difference in attitudes toward eLearn- 
ing between groups. None of the studies found more fa- 
vourable attitudes towards traditional learning. 

One study [52] that assessed the difference between tradi- 
tional learning and 2 different types of eLearning (8%) 
showed mixed results. The comparison between the tradi- 
tional learning group and the eLearning group with no in- 
teraction (ie, offline eLearning cases with no tests) showed 
that statistically significantly more students would recom- 
mend eLearning group with no interaction. However, the 
comparison between the control and the eLearning group 
with interaction (ie, cases with multiple choice and free- 
text questions) did not show a statistically significant dif- 
ference [52]. 

Students' satisfaction was assessed in 7 RCT studies [40, 
48,49,52,53,66,71] and 2 cRCT studies [45,54]. 

Out of 9 studies looking at the level of students' satisfac- 
tion, 5 (56%) studies [40,48,49,54,71] found a significant- 
ly greater proportion of students who were satisfied among 
those exposed to eLearning as compared to those exposed 
to traditional learning. Students' satisfaction was based on 
questionnaires, surveys and global perceptions of satisfac- 
tion. 

Two of the studies [52,53] showed higher satisfaction lev- 
els for students assigned to traditional learning groups. 

One (11%) study [45] did not detect any significant differ- 
ence while another study (11%) [66] did not test for sig- 
nificant differences and there were no clear trends in terms 
of 1 intervention group being superior to another. 

Comparison of different types of offline 
eLearning against each other 

Nine (18%) [43,44,46,47,56,59,61,62,68] of the included 
studies compared the effectiveness of various modes of of- 
fline eLearning against each other. 

Students' knowledge. Seven (78%) studies [43,44,47, 
56,61,62,68] compared various forms of offline eLearning 
and their effects on knowledge. A study [43] comparing 
the effectiveness of 3D vs 2D images of the larynx project- 
ed on a computer screen demonstrated higher test scores 
for students assigned to view 2D images. Another study 
[61] assessing the effectiveness of an actual video of oph- 
thalmic procedures vs actual video supplemented with 3D 
video demonstrated higher scores on theoretical knowl- 
edge for the group assigned to 3D video. 

One study [56] comparing 2 types of eLearning for teach- 
ing a module on leukaemia found that the more interactive 
eLearning intervention including questions resulted in sta- 
tistically significantly higher mean percentage scores on the 
post— test on leukaemia compared to the more passive in- 
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tervention group who only saw text and had no questions 
to answer. 

No differences were found in 3 studies [44,47,68] compar- 
ing different eLearning modalities with each other. Two of 
the studies [44,47] compared groups of eLearning with dif- 
ferent levels of student interaction with each other, where- 
as 1 group received no intervention. The third study [68] 
compared the effects of 2 versions of a program, 1 with 
animations and 1 with static graphics. 

One study [62] showed mixed findings, with 1 offline eL- 
earning mode exhibiting superior results with respect to a 
particular knowledge test and another offline eLearning 
mode exhibiting better results with respect to a different 
knowledge test. 

Students' skills. Skill acquisition was assessed in 3 (33%) 
[46,47,59] of the 9 studies which compared different of- 
fline eLearning modalities. Out of the 3 studies which as- 
sessed skill, 1 study [47] demonstrated better skill acquisi- 
tion with the use of a particular mode of offline eLearning 
over other modes. That study investigated the effects of 3 
different methods of manipulating contents for learning 
abdominal examination: click, watch and drag. Their re- 
sults showed that students who were able to use the mouse 
to trigger animated demonstrations ('click') performed bet- 
ter in auscultation than those who were in a more passive 
learning group where students only had control over the 
pace of the presentation ('watch'). The same group ('click') 
outperformed students who were in a more active learning 
group where students were able to drag tools in motions 
simulating actual performance of the task ('drag') in terms 
of abdominal palpation and additional manoeuvres. In ad- 
dition, more students in the 'drag' and 'click' groups cor- 
rectly diagnosed a simulated patient as having appendicitis 
than students in the 'watch' group. 

Two studies [46,59] failed to demonstrate any difference in 
skill acquisition between eLearning modes. 

Students' satisfaction and attitudes towards eLearning. 

Prinz et al. [61] and Morgulis et al. [56] were the only stud- 
ies amongst the 9 studies comparing different eLearning 
modalities that assessed attitudes towards eLearning. The 
study by Prinz et al. showed that the students in the 3D 
group rated the learning aid in the 3D group more useful 
compared to the control group students' rating of the learn- 
ing aid available in the control group and the difference was 
statistically significant. Intelligibility for glaucoma surgery 
and improvement of spatial ability both received statisti- 
cally significantly more positive responses in the 3D group 
compared to the control group. However, no difference was 
found for intelligibility for cataract surgery [61]. Similarly, 
the study by Morgulis et al. [56] that compared the use of 
existing online resources with a purpose-built, targeted eL- 
earning module on leukaemia for medical students dem- 



onstrated an overwhelmingly positive response from stu- 
dents assigned to the targeted module. 

Three (33%) studies [43,61,62] compared the effects of 
different eLearning modes on students' satisfaction. The 
study by Prinz et al. [61] , earlier cited for favourable results 
of 3D over 2D learning of ophthalmic procedures on 
knowledge, reported greater student satisfaction with the 
3D video. Although Hu et al.'s study [43] found that knowl- 
edge gain was higher for the 2D vs 3D learning group, en- 
joyment was higher in students assigned to 3D computer 
models. A study [62] which compared the effectiveness of 
a linear vs branched format for computer tutorials demon- 
strated that while the layout did not make a difference to 
their gain in ability, students in the linear group were slight- 
ly less likely to rate the tutorial as "valuable." 

DISCUSSION 

Our findings suggest that offline eLearning is at least equiv- 
alent, possibly superior to traditional learning in terms of 
students' knowledge, skills, and satisfaction and attitudes 
towards eLearning. Unfortunately, no studies evaluated im- 
pact on learners' professional attitudes towards patients. 
Eleven of the 33 studies testing knowledge gains found sig- 
nificantly higher gains in the eLearning intervention groups 
compared to traditional learning, whereas 2 1 did not de- 
tect significant differences or found mixed results. The re- 
maining study did not test for differences. Eight of the 13 
studies testing skill gains detected significantly higher gains 
in students allocated to the eLearning intervention, whilst 
5 of the studies did not find statistically significant differ- 
ences between the intervention and control group. Gener- 
ally no differences in attitudes or preference of eLearning 
over traditional learning were observed, nor between dif- 
ferent modes of offline eLearning. 

Studies varied considerably in terms of type of eLearning 
(ie, full eLearning vs blended learning), the content, deliv- 
ery channels, duration and frequency of exposure to the 
intervention, measures of outcomes, type of degrees, and 
seniority of students. For this reason, we did not calculate 
overall summary effect estimates. The majority of studies 
focused on full offline eLearning, whereas blended learn- 
ing was used in fewer studies. Although the majority of 
studies comparing offline eLearning with traditional learn- 
ing focused on seemingly similar offline eLearning pro- 
grams, the extent of interaction they provided varied from 
a simple PDF file [5 1 ,54] on a PDA as a learning aid when 
learning how to do drug calculations [39] to software with 
quizzes and other interactive features [47] . The duration of 
exposure to the eLearning interventions and the time from 
completion of the eLearning intervention until knowledge 
or skills were measured ranged from 12 minutes [46] to 1 
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year [58] . The complexity of the eLearning modalities also 
varied. However, apart from 1 study that used a PDA with 
software that could function without the internet [39] all 
studies used computers. 

The participants of the identified trials are representative 
of the intended population of students enrolled in under- 
graduate, health-related university degrees, and we expect 
that our results also apply to other similar university de- 
grees. However, only 5 [49,51,60,65,69] of the 49 includ- 
ed studies were conducted in low- to middle-income 
countries, none of which in the Mediterranean and African 
regions. Because we focused on offline eLearning that does 
not require internet access, the limited availability of data 
from developing countries does not limit the scope of the 
review in terms of the technology studied. However, due 
to the fact that computer literacy and cultural factors may 
determine the overall effects of eLearning on all domains 
we studied, it is possible that our conclusions on effective- 
ness are not applicable to all countries and settings. 

Over 50% of the studies [28,31,33,34,38,41-44,47,56, 
57,62,63,67-69,71,75] asked students whether they 
would be keen to participate in a trial on eLearning. The 
resulting study participants are thus likely to be more eager 
to use the eLearning interventions, which might have re- 
sulted in more favorable assessment of this educational ap- 
proach. Indeed, among the studies showing positive effects 
of eLearning, 4 studies [28,38,63,64] had a high risk and 
6 studies [27,36,39,40,50,73] had an unclear risk of vol- 
unteer bias. 

Our results are in line with the majority of the existing lit- 
erature. A review on online eLearning that we prepared in 
parallel also showed that the effects of online eLearning 
were equivalent, possibly superior to traditional learning. 
Likewise, a systematic review of 12 RCTs on computer- 
aided learning in dental education including both under- 
graduate students and dentists reported that statistically 
significant differences were not detected in the majority of 
studies comparing eLearning and traditional learning [21] . 
Another review of 12 randomized studies [20] concluded 
that the efficacy of computer-aided learning is reasonably 
well established. However, these authors also stressed that 
most of the included studies had methodological issues, 
eg, lack of power, attrition and a high risk of contamina- 
tion. These methodological issues were still present in the 
studies we included in our review despite it being pub- 
lished a decade later. 

There were also some differences between our results and 
the existing literature. Thirteen of the 14 included non- 
randomised controlled trials on the effect of computer- 
based instruction on knowledge and attitudes towards eL- 
earning of health professions students favoured eLearning 
over traditional methods in another review [86] . Out of the 



4 studies which compared students' attitudes towards the 
intervention, 3 demonstrated that computer-based in- 
struction students had more positive attitudes towards 
their instructional method than students exposed to con- 
ventional teaching [86] . Our findings were less positive to- 
wards offline eLearning and generally showed no difference 
in knowledge and attitudes between the intervention 
groups. This might potentially be explained by a larger 
presence of studies that did not blind the outcome assess- 
ment in the aforementioned review [86] . This could have 
resulted in students feeling more obliged to answer posi- 
tively. In addition, the review assessed the subjective out- 
comes of attitudes and satisfaction, the assessment of which 
was very heterogeneous in the included studies [86] , 
whereas we only assessed the results regarding students' 
satisfaction and attitude that dealt with the difference be- 
tween eLearning and traditional learning to keep the results 
as homogeneous as possible. Another systematic review 
[12] of 7 studies in allied health professions, medicine and 
nursing students reported that in all but 1 of the studies 
improvement in students' competencies, clinical skills, 
self-efficacy and clinical reasoning was seen when blended 
learning was used. This review included a very heteroge- 
neous sample of studies with both online and offline blend- 
ed learning. It included both controlled trials and trials that 
were not. Also, this review excluded all studies that did not 
report methods or results sufficiently or properly [12] . Our 
review yielded a less positive conclusion, perhaps because 
we considered all studies regardless of quality to assess the 
full body of evidence. Furthermore, we had a more com- 
prehensive search strategy allowing us to review a much 
larger number of studies. Because of these differences and 
the differences in topic, it is therefore not surprising that 
we reached different conclusions. 

Our study has many strengths. First, we optimised the 
probability of identification of all relevant literature by con- 
ducting our search using sensitive search strategy, multiple 
recognised literature databases without imposing language 
restrictions as well as by screening references of the select- 
ed articles. To enhance data quality, every identified article 
was screened by 2 people independently, and their results 
were compared. The same applied to the data extraction of 
the selected articles, which was enhanced by using a stan- 
dardised form for recording. The distinction between un- 
dergraduate and postgraduate education, and the focus on 
the former, increased the applicability of our results. The 
learning process at postgraduate level tends to be different, 
involving bedside learning and more in— depth exploration 
of the content. Additionally, patient outcomes are usually 
used as a proxy measure of the effectiveness of educational 
interventions in postgraduate education. An additional 
strength of our report was that our search resulted in the 
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inclusion of both developed and developing countries. Fi- 
nally, we followed the preferred reporting items for system- 
atic reviews and meta-analyses (PRISMA), a framework 
tool used to set the minimum evidence-based items to be 
included when conducting and writing systematic reviews. 
Doing so and by using the Cochrane methodology maxi- 
mized the completeness, transparency and accountability 
of our reporting of findings. 

Despite its strengths, our systematic review also has some 
limitations. First, we were unable to identify unpublished 
studies. Second, we were unable to consider the pedagog- 
ical approach in more detail mainly because of the incom- 
plete reporting of pedagogical methods within the includ- 
ed studies and because we did not request information on 
its details from the study authors. Third, our classification 
into offline eLearning and online eLearning and the other 
3 categories is pragmatic and not an established classifica- 
tion. Other authors may suggest other groupings. Howev- 
er, eLearning remains a recent field in which the defini- 
tions, concepts, evaluation tools and measures still lack 
consensus [87]. Some of the studies categorised and anal- 
ysed as offline eLearning were using eLearning interven- 
tions that were downloaded from WebCT Blackboard or 
sent to the students via email. This could be considered 
online eLearning, however, since the eLearning component 
could function fully offline and to avoid too much hetero- 
geneity between the studies, we classified it as offline. Also, 
the mode of delivery of the eLearning material could have 
been replaced by an offline one (eg, CR-ROM) and could 
therefore be used in areas with limited internet access. 

Finally, our choice to include articles from 2000 onwards 
only could be challenged. However the choice of 2000 can 
be justified by a rise in the interest in eLearning illustrated 
in part by several national and international reports and 
publications on the topic from this year onwards. These 
more recent reports are likely to have used more modern 
forms of eLearning than older reports and are thus time- 
lier, especially considering technological developments. 

Furthermore, there were some limitations to the evidence 
that was available and included in this review. The lack of 
or insufficient reporting of results in some of the included 
studies resulted in the restricted level of detail in the anal- 
ysis of certain outcomes of interest. Another important lim- 
itation to the evidence included is the lack of clarity of 1 or 
more aspects of the methodology used in the majority of 
the included trials and in some cases the occurrence of at- 
trition. Although we contacted authors to obtain the miss- 
ing information, some authors did not reply to our request 
and others did not know the answers. The lack of or insuf- 
ficient reporting of methods and results lead to an inability 
to draw a robust conclusion allowing for generalisation to 
all undergraduate students around the world due to the 



study selection process and the limitations of the included 
studies. 

We were unable to assess the cost-effectiveness of eLearn- 
ing vs traditional learning because none of the identified 
studies formally assessed it. The 6 articles [27,34,35,53, 
68,70] that mentioned economic elements such as hours 
spent developing the program suggested that eLearning 
modules cost more to develop than using already estab- 
lished traditional learning methods, but also highlight that 
this can be done with limited resources. 

None of the studies specifically addressed adverse effects 
of eLearning. This may be because potentially negative ef- 
fects of eLearning that are regularly cited [24] focus on 
loneliness and depression, which could be regarded to be 
an aspect of students' satisfaction and attitudes. Potential 
reduced efficacy and effectiveness of eLearning would have 
been evaluated as part of the assessment of skills and 
knowledge. 

In summary, the findings from this systematic review sug- 
gest that offline eLearning is similar to traditional learning 
in terms of knowledge and skill acquisition and that it is 
possibly superior to traditional learning. In addition, they 
offer a more convenient, and more cost— effective, alterna- 
tive to facilitate competency development and the training 
of health care professionals around the globe. 

Our results indicate that students were more favourable 
towards the eLearning interventions. However, due to a 
high risk of bias these results should be interpreted with 
caution. 

Implications for policy makers 

This systematic review indicates that offline eLearning is 
likely to be as effective as traditional learning, possibly su- 
perior and this presents a potential incentive for policy 
makers to encourage the development of offline eLearning 
curricula. These offline eLearning programs could poten- 
tially help address the health care worker shortage by con- 
tributing to greater access to education and training as part 
of scaling up the education of health workers especially in 
the developing world where internet access is limited and 
the need for an increase in the number of health profes- 
sionals is greatest. However, there are still barriers (eg, com- 
puter access and access to eLearning material) that need to 
be overcome and this could be helped by changing policies 
and facilitating investments in ICT. 

Implications for educational institutions 

Many eLearning programs were developed by local enthu- 
siasts within universities and this review showed that these 
programs were likely to be effective in terms of knowledge 
and skills acquisition. Therefore, educational institutions 
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should encourage such enthusiasts who wish to develop 
eLearning materials to improve the learning experience and 
knowledge and skills acquired by their students. Despite 
the fact that a robust conclusion on whether or not eLearn- 
ing is superior to traditional learning could not be drawn 
we believe that educational institutions should not refrain 
from investing in offline eLearning material. 

Implications for future research 

Offline eLearning is still likely to be a key player in educa- 
tion in the next decade where technology in education is 
expected to be used more and more and therefore research- 



ers should continue to investigate the effects of this inter- 
vention on knowledge, skills and students' satisfaction and 
attitudes especially in developing countries. Future indi- 
vidual studies should continue to improve the methodol- 
ogy (eg, avoid contamination and volunteer bias) with 
which the eLearning intervention is investigated and report 
their study according to the CONSORL guidelines. 

Furthermore, we suggest that a well-defined and common- 
ly used taxonomy for the different types and aspects of eL- 
earning should be developed and employed in future re- 
search to enable easier comparison of different eLearning 
studies. 
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